Agents. What did we think about agents last semester? We've already seen this. We have

an environment, we have sensors, we have actuators, and that's essentially all we need for an

agent. And we're interested in what's in there. Different way of seeing the same picture,

only we have a lot more space here to fill in stuff. So this is our agent schema. And

I'm going to go over, what I'm going to do is go into the important environment, the

important parts, and to describe an agent, we have this P's schema, which is what is

the performance measure? How are we going to judge whether our agent does well? What

is the environment? Where do we expect our agent to work well? The actuators and the

sensors. And if you look at something like an internet shopping agent, this would be

something like you would have a performance measure, which you would judge it on, whether

it gets you a good price and quality, whether it actually gets the appropriate stuff. An

agent that gets me extremely good and cheap children's pants doesn't help me very much,

so it did badly if it bought those for me. And of course, efficiency might be something

you're interested in. The environment, of course, is the current and future World Wide

Web with all its online stores and so on, but not the past. We wouldn't be particularly

interested in whether it performed well on the past internet. You have actuators, which

are displays typically. It can follow a URL by getting a new web page. It can fill in

the form typically. The sensors would be essentially reading HTML pages. That's all it needs. It

doesn't need a sense of smell or those kind of things. So that's how we would think of

things. And you can already see that there's quite a wide variety of things you can do.

One of the main variables we've looked at in a little bit more detail is the environment.

We started with the easy environments. The question, of course, is whether our environment

is fully observable, which means in the Wumpus world, we had an environment that wasn't because

the cave is dark. We cannot see where the Wumpus is, even if we wanted to. So there

we already had a partially observable environment. The question is whether it's deterministic,

which means whether the state of the environment is fully determined by the current state of

the environment and, of course, our actions. So the typical thing is if I have an environment

where I drop something onto my foot, will it actually hurt after a while? In a non-deterministic

environment, you might have something intervening. Strong wind, and I drop this on my foot, it

never reaches my foot. Then there's, of course, the question whether we have episodic environments.

Degenerative, dynamic environments are things like playing chess. You have episodes, and

all you need to do is everything you're really interested in is what happens. What's the

state after one episode? You're not interested in the detail, whether your opponent just

does this with their piece or does that. It might annoy you, but it doesn't really change

the way the chess piece goes. Your typical environments aren't, by quantifying time,

can sometimes make things vaguely episodic, but most real-time environments aren't. There's

the question whether an environment is dynamic or static. In a static environment, you're

actually the only one who can change the environment. Typically in a game, and that's part of the

fun, of course, is that somebody else also changes the environment. Playing soccer, where

you're the only one who's acting may be fun for a while, but there's no big challenge.

So you have dynamic and static environments. Then of course, sometimes we have multiple

agents, and so on, and we have discrete and continuous environments. Last semester, we've

essentially chickened out and took the easy choice here. This semester, we're not going

to chicken out that much anymore, and we're going to look at what that means for us. Here

I have a couple of environment types that you might remember.

We looked at a couple of agent architectures. We had the simple reflex agent, which essentially

senses the world and has a model of the world. It knows what the world is like by looking

at it, by sensing it. Then it just basically has some condition-action rules that if it's

hot, get into the shade. If somebody screens at me, run away. These kind of things. Simple

bacteria really follow this. That's a very simple, easy to make agent. It's just not